Testing Data Transformations in MapReduce Programs

MapReduce is a parallel data processing paradigm oriented to process large volumes of information in data-intensive applications, such as Big Data environments. A characteristic of these applications is that they can have different data sources and data formats. For these reasons, the inputs could contain some poor quality data that could produce a failure if the program functionality does not handle properly the variety of input data. The output of these programs is obtained from a number of input transformations that represent the program logic. This paper proposes the testing technique called MRFlow that is based on data flow test criteria and oriented to transformations analysis between the input and the output in order to detect defects in MapReduce programs. MRFlow is applied over some MapReduce programs and detects several defect

Automatic generation of assumptions for modular verification of software specifications

Model checking is a powerful automated technique mainly used for the verification of properties of reactive systems. In practice, model checkers are limited due to the state explosion problem. Modular verification based on the assume-guarantee paradigm mitigates this problem using a “divide and conquer” technique. Unfortunately, this approach is not automated, for the reason that the user must specify the environment model. In this paper, a novel technique is presented for automatically generating component assumptions based on the behaviour of the environment (the remainder of components of the systems). In a first phase, the environment of the component is computed using state space exploration techniques, and then the assumptions are generated as association rules of the component environment interface. This approach presents a number of advantages. Firstly, user assistance to specify assumptions is not necessary and assumption discharge is avoided. Secondly, the component assumptions are more restrictive and real, and therefore reduce the resources needed by the model checker. The technique is applied to the specification of a steam boiler syste

RETORCH: Resource-Aware End-to-End Test Orchestration

12th International Conference, QUATIC 2019, Ciudad Real, Spain, September 11–13, 2019Continuous integration practices introduce incremental changes in the code to both improve the quality and add new functionality. These changes can introduce faults that can be timely detected through continuous testing by automating the test cases and re-executing them at each code change. However, re-executing all test cases at each change may not be always feasible, especially for those test cases that make heavy use of resources thoroughly like End-to-End test cases that need a complex test infrastructure. This paper is focused on optimizing the usage of the resources employed during End-to-End testing (e.g., storage, memory, web servers or tables of a database, among others) through a resource-aware test orchestration technique in the context of continuous integration in the cloud. In order to optimize both the cost/usage of resources and the execution time, the approach proposes to (i) identify the resources required by the End-to-End test cases, (ii) group together those tests that need the same resources, (iii) deploy the tests in both dependency isolated and elastic environments, and (iv) schedule their parallel execution in several machine

Mutating database queries

A set of mutation operators for SQL queries that retrieve information from a database is developed and tested against a set of queries drawn from the NIST SQL Conformance Test Suite. The mutation operators cover a wide spectrum of SQL features, including the handling of null values. Additional experiments are performed to explore whether the cost of executing mutants can be reduced using selective mutation or the test suite size can be reduced by using an appropriate ordering of the mutants. The SQL mutation approach can be helpful in assessing the adequacy of database test cases and their development, and as a tool for systematically injecting faults in order to compare different database testing technique

Infrastructure-Aware Functional Testing of MapReduce Programs

2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), Vienna, 2016Programs that process a large volume of data generally run in a distributed and parallel architecture, such as the programs implemented in the processing model MapReduce. In these programs, developers can abstract the infrastructure where the program will run and focus on the functional issues. However, the infrastructure configuration and its state cause different parallel executions of the program and some could derive in functional faults which are hard to reveal. In general, the infrastructure that executes the program is not considered during the testing, because the tests usually contain few input data and then the parallelization is not necessary. In this paper a testing technique is proposed to generate different infrastructure configurations for a given test input data, and then the program is executed in these configurations in order to reveal functional faults. This testing technique is automatized by using a test engine and is applied to a case study. As a result, several infrastructure configurations are automatically generated and executed for a test case revealing a functional fault that is then fixed by the develope

Towards Ex Vivo Testing of MapReduce Applications

2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), 25-29 July 2017, Prague (Czech Republic)Big Data programs are those that process large data exceeding the capabilities of traditional technologies. Among newly proposed processing models, MapReduce stands out as it allows the analysis of schema-less data in large distributed environments with frequent infrastructure failures. Functional faults in MapReduce are hard to detect in a testing/preproduction environment due to its distributed characteristics. We propose an automatic test framework implementing a novel testing approach called Ex Vivo. The framework employs data from production but executes the tests in a laboratory to avoid side-effects on the application. Faults are detected automatically without human intervention by checking if the same data would generate different outputs with different infrastructure configurations. The framework (MrExist) is validated with a real-world program. MrExist can identify a fault in a few seconds, then the program can be stopped, not only avoiding an incorrect output, but also saving money, time and energy of production resource

Generating iInput documents for testing XML queries with ToXgene

Testing: Academic & Industrial Conference - Practice and Research Techniques (TAIC-PART). WindsorModern software technologies use the XML language as a preferred means of data interchange and representation, which can be accessed using XML queries. Testing these queries can be difficult because they use complex data as input in form of XML documents. This work addresses this problem by defining a query-aware procedure for design and generation of test input data for XML queries. The test process is based on the category-partition method and permits the identification of categories and choices in the XML query and schema in order to obtain test case specifications. These test case specifications are mapped to the input language of the ToXgene data generator and then XML documents are generated by running i

Use of micro CHP plants to support the local operation of electric heat pumps

Fig. 1. Global distribution of chytridiomycosis-associated amphibian species declines. Bar plots indicate the number (N) of declined species, grouped by continental area and classified by decline severity. Brazilian species are plotted separately from all other South American species (South America W); Mesoamerica includes Central America, Mexico, and the Caribbean Islands; and Oceania includes Australia and New Zealand. No declines have been reported in Asia. n, total number of declines by region. [Photo credits (clockwise from top left): Anaxyrus boreas, C. Brown, U.S. Geological Survey; Atelopus varius, B.G.; Salamandra salamandra, D. Descouens, Wikimedia Commons; Telmatobius sanborni, I.D.l.R; Cycloramphus boraceiensis, L.F.T.; Cardioglossa melanogaster, M.H.; and Pseudophryne corroboree, C. Doughty